The invention relates to the field of wide area, or, when imaged regions are all contiguous, panoramic imaging. The invention has particular utility for commercial security and surveillance but is readily used for other applications that require a low cost wide field of view device, particularly mid-wave and long-wave thermal infrared (MWIR, LWIR respectively).
There are at least three ways to obtain a wide area image, including a 360 degree panorama, with a focal plane array (FPA): 1) use multiple copies of the FPA each having its own lens and field of view, and assign each of the FPAs to a particular segment of the wide area; 2) use projection optics to map pixels around the periphery to a common FPA, typically imaging the periphery onto an annulus within the FPA; 3) move or scan the FPA and lens so as to use a limited field of view device (i.e., <360 degrees) to collect images in sequence that, when viewed together completely image the wide area, or in the case of a 360 degree panorama, comprise a 360 degree field of view.
The choice of one method over another is often driven by the tradeoff between cost, frame rate (images per second) and sensitivity (amount of energy needed for a good image). Presently, reliable and relatively low cost MWIR and LWIR FPAs, e.g., uncooled microbolometers, are readily available. These are sometimes used in scenario (1) above, but require many FPA's, thereby making this scenario costly and less reliable for commercial applications. When these devices are used for scenario (3), they are inefficient in the use of pixels and also require an expensive peripheral window, both of which push cost upwards. The use of uncooled microbolometer FPAs continuously scanning a wide area can provide a low cost means of imaging, but the imaging speed is too low for real-time applications, owing to the low sensitivity, e.g., D*, of such thermal devices (as compared to cooled photonic devices) and the typically low sampling rate required by pixel thermal mass constraints. Thus even with a relatively low cost technology (uncooled microbolometers) it is yet relatively expensive to obtain a 360 degree field of view.
Thus it is typical to use a relatively expensive sensor technology, e.g., cooled HgCdTe or something comparable, but use only a minimal size FPA, e.g., a few columns and a few hundred rows, so as to minimize the amount of expensive detector material and yet afford the opportunity to generate many resultant image pixels by using a simple, low cost mechanism—a continuous scan, constant velocity rotary mechanism. Typically, the hundreds of rows are used to image the vertical dimension and the few columns are used to capture the horizontal, rotating, dimension and many columns are joined (“stitched”) together. Furthermore, TDI (time delay and integration) techniques can be used to integrate many “looks” per column together so as to accumulate signal and overcome any shortfall in sensitivity, even while mitigating the effects of motion induced blur.
The resultant wide area imaging system can be very effective, but even with a good tradeoff in the choice of sensor technology and the number of pixels used, the overall solution has a relatively low mean time to failure which makes it expensive to maintain and the systems components are quite expensive, often much more expensive than the aforementioned multiple-FPA approach for uncooled microbolometers devices.
The objective of the invention then, is to produce a time series of wide area images, i.e., having a field of view greater than that of the sensor by itself, so that object detection and related data reduction activities ensue. A traditional panorama involves a set of adjacent images that have some overlap that permits edges to be merged seamlessly into a single composite image. Such a panorama is within the capability of the invention, but only sometimes generated by the invention. This is due to the desire to minimize time spent on image areas that have little interest, leaving more time (and therefore higher rates of imaging or coverage) for image areas that have elevated interest. For instance, in a perimeter surveillance scenario where a fence line is to be monitored, the area adjacent to, but outside, the fence, is typically of more interest than areas inside the fence, so an imaging system would provide higher performance for detecting activity outside the fence if more time could be spent imaging those regions. This desire for a high degree of flexibility and control in the distribution of the imagery generated leads to a requirement for computer control of both timing of sensor imaging and the motion profile. Because the motion of the sensor is under computer control, e.g., a direct drive servo motor driven by a real time motor controller with encoder feedback, in the preferred embodiment, the sensor motion profile can be arbitrarily determined—there is no pattern that cannot be commanded and generated on demand, adaptively when needed, within the range of velocities and positions enabled by specific choices of motor, amplifier, encoder and mass/inertia of the sensor assembly. The preferred embodiment uses a direct drive servo for the precision control it provides, but also for the longevity, consistency and reliability provided by the absence of gears, pulleys or otherwise friction-based mechanical assemblies. Such assemblies can be made to work with the invention, but will not be as broadly optimal as the direct drive servo motor implementation.